Skip to content

fix: detect codex model capacity errors as blocked - #2342

Closed
woodgear wants to merge 2 commits into
herdrdev:masterfrom
woodgear:fix/codex-capacity-blocked
Closed

fix: detect codex model capacity errors as blocked#2342
woodgear wants to merge 2 commits into
herdrdev:masterfrom
woodgear:fix/codex-capacity-blocked

Conversation

@woodgear

@woodgear woodgear commented Aug 5, 2026

Copy link
Copy Markdown

Summary

  • classify Codex model-capacity errors as blocked
  • limit matching to the bottom three non-empty lines so stale errors remain idle
  • publish the updated Codex detection manifest

Validation

  • captured the affected pane with herdr agent read <pane> --source detection --format text
  • stock Herdr 0.7.5 classified the captured screen as idle; the patched bundled manifest classifies it as blocked
  • just ci (Rust 1.96.1, Zig 0.15.2): 3,217 Rust tests, 19 integration asset tests, and 24 plugin marketplace tests passed
  • python3 scripts/agent_detection_manifest_check.py --require-website

@kangal-bot kangal-bot added the ai-review Trigger automated AI reviews for pull requests admitted by the PR gate label Aug 5, 2026
@coderabbitai

coderabbitai Bot commented Aug 5, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The Codex manifests now detect model-capacity messages as visible blockers within the bottom four non-empty lines. Tests cover blocked, active-spinner, and stale-activity states.

Changes

Codex capacity detection

Layer / File(s) Summary
Add the capacity-blocked rule
src/detect/manifests/codex.toml, website/agent-detection/codex.toml
Both Codex manifests add the model_capacity_blocked rule with priority 490. The rule matches capacity messages in the bottom four non-empty lines. Both manifests update their version and timestamp.
Validate blocked and activity states
src/detect/manifest/tests.rs
Tests verify visible blocked detection without activity, working state when an OSC spinner is active, and idle state when activity history is stale.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly describes the main change: detecting Codex model-capacity errors as blocked.
Description check ✅ Passed The description directly explains the Codex detection change, matching behavior, manifest updates, and validation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ecbd1553-4b96-4d3c-9778-7db59ece392f

📥 Commits

Reviewing files that changed from the base of the PR and between 15442a2 and 30b472c.

📒 Files selected for processing (3)
  • src/detect/manifest/tests.rs
  • src/detect/manifests/codex.toml
  • website/agent-detection/codex.toml

Comment thread src/detect/manifests/codex.toml
@greptile-apps

greptile-apps Bot commented Aug 5, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds Codex model-capacity detection and publishes the updated manifest, with regression tests for current, active, and stale screens.

  • Adds a blocked-state rule for the Codex capacity message.
  • Restricts matching to a bottom-of-screen non-empty-line region.
  • Mirrors the bundled manifest to the website publication copy.
  • Adds tests covering wrapped messages, active spinners, and stale history.

Confidence Score: 4/5

The stale one-line capacity boundary should be fixed before merging because it can incorrectly leave an idle Codex pane blocked.

The new four-line region includes a capacity error when exactly three newer non-empty lines follow it, and the blocked rule then outranks the static-title idle rule.

Files Needing Attention: src/detect/manifests/codex.toml, website/agent-detection/codex.toml, src/detect/manifest/tests.rs

Important Files Changed

Filename Overview
src/detect/manifests/codex.toml Adds the capacity blocker, but its four-line region retains a stale one-line error at the intended three-newer-line boundary.
website/agent-detection/codex.toml Correctly mirrors the bundled manifest, including the same stale-error boundary defect.
src/detect/manifest/tests.rs Adds useful regression coverage, but the stale case uses five non-empty lines and misses the four-line boundary.

Reviews (1): Last reviewed commit: "fix: harden codex capacity detection" | Re-trigger Greptile

id = "model_capacity_blocked"
state = "blocked"
priority = 490
region = "bottom_non_empty_lines(4)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Stale capacity error remains blocked

When a single-line capacity error is followed by exactly three newer non-empty lines, bottom_non_empty_lines(4) retains the stale error and the blocked rule outranks the static-title idle rule, causing an idle Codex pane to remain reported as blocked.

Knowledge Base Used: Agent detection, Ghostty integration, and configuration

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1


ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 50ee8a4b-d4fb-4cab-8085-54ae2cc520aa

📥 Commits

Reviewing files that changed from the base of the PR and between 30b472c and 53b1c38.

📒 Files selected for processing (3)
  • src/detect/manifest/tests.rs
  • src/detect/manifests/codex.toml
  • website/agent-detection/codex.toml
🚧 Files skipped from review as they are similar to previous changes (2)
  • website/agent-detection/codex.toml
  • src/detect/manifest/tests.rs

id = "model_capacity_blocked"
state = "blocked"
priority = 490
region = "bottom_non_empty_lines(4)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -eu
printf '%s\n' '--- manifest ---'
cat -n src/detect/manifests/codex.toml
printf '%s\n' '--- relevant tests ---'
sed -n '700,780p' src/detect/manifest/tests.rs
printf '%s\n' '--- related rules and helpers ---'
rg -n -C 3 'bottom_non_empty_lines|capacity|codex' src/detect
printf '%s\n' '--- repository changes ---'
git diff --stat
git status --short
printf '%s\n' '--- validation recipes ---'
if command -v just >/dev/null 2>&1; then just --list 2>/dev/null | sed -n '1,120p'; else echo 'just unavailable'; fi

Repository: herdrdev/herdr

Length of output: 50370


🏁 Script executed:

#!/bin/bash
set -eu
printf '%s\n' '--- capacity tests ---'
sed -n '760,790p' src/detect/manifest/tests.rs
printf '%s\n' '--- region implementation ---'
sed -n '1240,1345p' src/detect/manifest.rs
printf '%s\n' '--- region tests ---'
sed -n '520,550p' src/detect/manifest/tests.rs
printf '%s\n' '--- project requirement references ---'
rg -n -i -C 3 'bottom three|three-line|three lines|capacity error|model capacity|at capacity|codex' \
  README.md .github src tests 2>/dev/null | sed -n '1,240p'
printf '%s\n' '--- just recipe definition ---'
if [ -f justfile ]; then rg -n -C 3 '^check|check:' justfile; else echo 'justfile not found'; fi

Repository: herdrdev/herdr

Length of output: 22276


🏁 Script executed:

#!/bin/bash
set -eu
python3 - <<'PY'
import re

screen = (
    "⚠ Selected model is at capacity. Please\n"
    "  try a different model.\n\n"
    "› Summarize recent commits\n\n"
    "gpt-5.6-sol high · /work · Ready\n"
)
pattern = re.compile(
    r"(?m)^⚠ Selected\s+model\s+is\s+at\s+capacity\.\s+Please\s+"
    r"a\s+different\s+model\.$"
)

def bottom_non_empty_lines(content, count):
    lines = content.splitlines()
    selected = [i for i, line in reversed(list(enumerate(lines)))
                if line.strip()][:count]
    return "" if not selected else "\n".join(lines[min(selected)]) + "\n" if False else \
        "\n".join(lines[min(selected):]) + "\n"

for count in (3, 4):
    region = bottom_non_empty_lines(screen, count)
    print(f"count={count}")
    print("region_lines=", repr(region.splitlines()))
    print("capacity_match=", bool(pattern.search(region)))
PY
printf '%s\n' '--- exact manifest diff ---'
git diff -- src/detect/manifests/codex.toml src/detect/manifest/tests.rs

Repository: herdrdev/herdr

Length of output: 517


🏁 Script executed:

#!/bin/bash
set -eu
python3 - <<'PY'
import re

screen = (
    "⚠ Selected model is at capacity. Please\n"
    "  try a different model.\n\n"
    "› Summarize recent commits\n\n"
    "gpt-5.6-sol high · /work · Ready\n"
)
pattern = re.compile(
    r"(?m)^⚠ Selected\s+model\s+is\s+at\s+capacity\.\s+"
    r"Please\s+try\s+a\s+different\s+model\.$"
)

def bottom_non_empty_lines(content, count):
    lines = content.splitlines()
    indices = [i for i, line in reversed(list(enumerate(lines)))
               if line.strip()][:count]
    return "" if not indices else "\n".join(lines[min(indices):]) + "\n"

for count in (3, 4):
    region = bottom_non_empty_lines(screen, count)
    print(f"count={count}")
    print("region_lines=", repr(region.splitlines()))
    print("capacity_match=", bool(pattern.search(region)))

stale = (
    "⚠ Selected model is at capacity. Please try a different model.\n\n"
    "• Explored\n"
    "└ Read Cargo.toml\n\n"
    "› Summarize recent commits\n\n"
    "gpt-5.6-sol high · /work · Ready\n"
)
for count in (3, 4):
    region = bottom_non_empty_lines(stale, count)
    print(f"stale_count={count}", "capacity_match=", bool(pattern.search(region)))
PY

Repository: herdrdev/herdr

Length of output: 560


Match the required three-line bottom window.

bottom_non_empty_lines(4) matches a warning with three newer non-empty lines. Use bottom_non_empty_lines(3) and update the wrapped fixture, or change the requirement to four lines. Run just check.

Source: Coding guidelines

@ogulcancelik

Copy link
Copy Markdown
Collaborator

thanks for the PR. after looking at Codex’s behavior more closely, I don’t think we should detect this as blocked.

Codex ends the turn and returns to idle when the capacity error occurs. That seems reasonable, and distinguishing a current error from stale terminal history requires brittle screen matching. Herdr’s attention state doesn’t need to cover every terminal error.

I’m going to close this PR, but I appreciate you raising the case and helping us investigate it.

@woodgear
woodgear deleted the fix/codex-capacity-blocked branch August 6, 2026 03:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-review Trigger automated AI reviews for pull requests admitted by the PR gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants